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Recently, Zhang and Van Breugel introduced the notion of a progress measure for a probabilistic 
model checker. Given a linear-time property and a description of the part of the system that 
has already been checked, the progress measure returns a real number in the unit interval. The real 
number captures how much progress the model checker has made towards verifying (j) . If the progress 
is zero, no progress has been made. If it is one, the model checker is done. They showed that the 
progress measure provides a lower bound for the measure of the set of execution paths that satisfy (j). 
They also presented an algorithm to compute the progress measure when is an invariant. 

In this paper, we present an algorithm to compute the progress measure when is a formula of a 
positive fragment of linear temporal logic. In this fragment, we can express invariants but also many 
other interesting properties. The algorithm is exponential in the size of and polynomial in the size 
of that part of the system that has already been checked. We also present an algorithm to compute a 
lower bound for the progress measure in polynomial time. 

1 Introduction 

Due to the infamous state space explosion problem, model checking a property of source code that 
contains randomization often fails. In many cases, the probabilistic model checker simply runs out of 
memory without reporting any useful information. In UT], Zhang and Van Breugel proposed a progress 
measure for probabilistic model checkers. This measure captures the amount of progress the model 
checker has made with its verification effort. Even if the model checker runs out of memory, the amount 
of progress may provide useful information. 

Our aim is to develop a theory that is applicable to probabilistic model checkers in general. Our 
initial development has been guided by a probabilistic extension of the model checker Java PathFinder 
(JPF) ||9l- This model checker can check properties, expressed in linear temporal logic (LTL), of Java 
code containing probabilistic choices. 

We model the code under verification as a probabilistic transition system (PTS), and the systematic 
search of the system by the model checker as the set of explored transitions of the PTS. We focus on 
linear-time properties, in particular those expressed in LTL. The progress measure is defined in terms 
of the set of explored transitions and the linear-time property under verification. The progress measure 
returns a real number in the interval [0, 1]. The larger this number, the more progress the model checker 
has made with its verification effort. 

Zhang and Van Breugel showed that their progress measure provides a lower bound for the measure 
of the set of execution paths that satisfy the linear-time property under verification. If, for example, the 
progress is 0.9999, then the probability that we encounter a violation of the linear-time property when 
we run the code is at most 0.0001. Hence, despite the fact the model checker may fail by running out 
of memory, the verification effort may still be a success by providing an acceptable upper bound on the 
probability of a violation of the property. 
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The two main contributions of this paper are 

1 . a characterization of the progress measure for a positive fragment of LTL. This fragment includes 
invariants, and most examples found in, for example, (T, Section 5.1] can be expressed in this 
fragment. This characterization forms the basis for an algorithm to compute the progress measure. 

2. a polynomial time algorithm to compute a lower bound for the progress measure for the positive 
fragment of LTL. The lower bound is tight for invariants, that is, this algorithm computes the 
progress for invariants. 

2 A Progress Measure 

In this section, we review some of the key notions and results of [llj. We represent the system to be 
verified by the probabilistic model checker as a probabilistic transition system. 

Definition 1 A probabilistic transition system is a tuple (S,r,Af,so, source, target, prob, label) consist- 
ing of 

• a countable set S of states, 

• a countable set T of transitions, 

• a set AP of atomic propositions, 

• an initial state sq, 

• a function source : T — )• 5, 

• a function target :T ^ S, 

• a function prob : T — t- (0, 1], and 

• a function label : S — t- 2^^ 
such that 

• So £ S and 

• for all s £ S, L{prob(f) | source(f) = 5} = 1. 
Example 2 The probabilistic transition system depicted by 




has three states and six transitions. In this example, we use the indices of the source and target to name 
the transitions. For example, the transition from sq to S2 is named to2- Given this naming convention, 
the functions source,5^ and targety are defined in the obvious way. For example, sourcej^(fo2) = so and 
targety(fo2) = ^2- The function prohy can be easily extracted from the above diagram. For example, 
prob,y^(fo2) = 2- states are labelled with the atomic proposition a and the states si and S2 are also 
labelled with the atomic proposition b. Hence, for example, labely (s2) = {a,fe}. 
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Instead of (5, r,AP,50) source, target, prob, label) we usually write =y and we denote, for example, 
its set of states by Sy. We model the potential executions of the system under verification as execution 
paths of thePTS. 

Definition 3 An execution path of a PTS 5^ is an infinite sequence of transitions t\t2 ■ ■ ■ such that 

• for all i > 1, ti G Ty, 

• source.y (ti ) = SQy, and 

• for all i > 1, target^(?,) = source,^ (fi+i). 
The set of all execution paths is denoted by Exec y. 

Example 4 Consider the PTS of Example^ For this system, 102122"', foifn^sa'", <^nd foifiof02?22'" are 
examples of execution paths. 

To define the progress measure, we use a measurable space of execution paths. We assume that the 
reader is familiar with the basics of measure theory as can be found in, for example, |3|. Recall that a 
measurable space consists of a set, a a-algebra and a measure. In our case, the set is Execy. The a- 
algebra £ y is generated from the basic cylinder sets defined below. We denote the set of finite prefixes 
of execution paths in Execy by pref(Exec,y). 

Definition 5 Let e G pref(Exec,y). Its basic cylinder set By is defined by 

By^ = {e £ Execy \ e is a prefix ofe'}. 
The measure lJ.y is defined on a basic cylinder set B'^"'" by 

^y{By'-)= n P^ohy{ti). 

l<i<n 

The measurable space (Exec ^ ,E.j^ ,jUj^) is a sequence space as defined, for example, in (F, Chapter 2]. 

The verification effort of the probabilistic model checker is represented by its search of the PTS. The 
search is captured by the set of transitions that have been explored during the search. 

Definition 6 A search of a PTS .5^ is a finite subset ofTy. 

Example 7 Consider the PTS of Example^ The sets®, {foi}. {^02}. {to\,to2} <^nd {t()\,to2,tiQ,t\3, 122,(33} 
are examples of searches. 

A PTS is said to extend a search if the transitions of the search are part of the PTS. We will use this 
notion in the definition of the progress measure. 

Definition 8 The PTS y extends the search T of the PTS y if for all t £ T, 

• t G Ty,, 

• ^oy = ^oyi, 

• sourcey/(f) = source.y (f), 

• targety'{t) = largely (t), 

• probyi{t) =proby{t), 

• labely/(sourcey/(f)) = labely(sourcey (f)), and 
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• label^/ (target^/ (f)) = label,y(target^'(f)). 
Example 9 Consider the PTS of Example^and the search {foi , fo2}- The PTS 




extends the search. 

Since the PTSs we will consider in the remainder of this paper all extend a search T of a PTS y, we 
write ^0 instead of s^y to avoid clutter. PTSs that extend a particular search give rise to the same set of 
execution paths if we restrict ourselves to those execution paths that only consist of transitions explored 
during the search. 

Proposition 10 If the PTS 5^' extends the search T of the PTS .5^, then 

(a) r* npref(Exec.y) = T* npref(Exec.y^/) and 

(b) T'^DExecy = nExec^/. 

PTSs that extend a particular search also assign the same measure to basic cylinder sets of prefixes 
of execution paths only consisting of transitions explored during the search. 

Proposition 11 If the PTS 5^' extends the search T of the PTS 5^, then iJ.y{By) = pLyi{B''y,)for all 
e G r*npref(Execy). 

The function labels assigns to each state the set of atomic propositions that hold in the state. This 
function is extended to (prefixes of) execution paths as follows. 

Definition 12 The function truce y : Execy — )■ (2'*^ *' )® is defined by 

trace,y'(fif2 • • •) = label^(source,y (fi))label^'(source,y (f2)) • • ■ 

The function tructy : pref(Exec.y) — (lA^-^)* is defined by 

trace,y(fi . . = label.y (source,y (?i)) . . .label j/(source,y(f„))labelj? (target jy'(f„)) 

Example 13 Consider the PTS of Example^ 

M^eey(t(ntTi') = {a]{a,b]'^ 
trace.y'(f()ifi3f33'") = {a]{a,b]{a}"' 
trace,y'(foifio?02f22'") = {a}{a,b}{a}{a,b}'^ 

For the definition of linear-time property and the satisfaction relation |= we refer the reader to, for 
example, El Section 3.2]. Based on these notions, we define when an execution path of a PTS satisfies a 
linear-time property. 

Definition 14 The satisfaction relation \=y is defined by 

e \=y <p if tra.cey(e) \= (j). 
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For PTSs that extend a particular search, those execution paths that only consist of transitions ex- 
plored by the search satisfy the same linear-time properties. 

Proposition 15 Let be a linear-time property. If the PTS 5^' extends the search T of the PTS 5^, then 
e \=y' (p iff e \=yi <pfor all e € Pi Exec y. 

Proof Since J/^' extends T of J^, trace,y' (e) = tiaceyi{e) for all e G T'" HExecy. □ 

Next, we introduce the notion of a progress measure. Given a search of a PTS and a linear-time 
property, it captures the amount of progress the search of the probabilistic model checker has made 
towards verifying the linear-time property. 

Definition 16 Let the PTS S^' extend the search T of PTS 5^ and let ^ be a linear-time property. The 
set SS^^, (T) is defined by 

^^,(r) = [jiB'y,, I e e r* AVe' e b'^, e' \=y, (p }. 

The set (T) is the union of those basic cylinder sets B^, the execution paths of which satisfy the 
linear-time property 0. Hence, Sy., does not contain any execution paths violating 0. The set =^^,(7) is 
measurable, as shown in ifTTl Proposition 1]. Hence, the measure Hyi assigns it a real number in the unit 
interval. This number represents the "size" of the basic cylinder sets that do not contain any violations 
of <p . This number captures the amount of progress of the search T verifying , provided that the PTS 
under consideration is However, we have no knowledge of the transitions other than the search. 
Therefore, we consider all extensions of T and consider the worst case in terms of progress. 

Definition 17 The progress of the search T of the PTS ,y of the linear-time property (j) is defined by 

pmgy{T,^) = inf I y, (^J-^r)) | y' extends T of . 

Example 18 Consider the PTS 5^ of Example^and the linear temporal logic formulae Da, ()a, ()b and 
Q)b. In the table below, we present the progress of these properties for a number of searches. 
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In ifTTl Theorem 1], Zhang and Van Breugel prove the following key property of their progress 
measure. They show that it is a lower bound for the probability that the linear-time property holds. 

Theorem 19 Let T be a search of the PTS .5^ and let ^ be a linear-time property. Then 

progj^(r, 0) < \lyi{e £ Execy \ e ^y <p }). 

The setting in this paper is slightly different from the one in ifTTI . In this paper we assume that PTSs 
do not have final states. This assumption can be made without loss of any generality: simply add a self 
loop with probability one to each final state. 



38 



Measuring Progress of Probabilistic LTL Model Checking 



3 Negation and Violations 

In this section, we consider the relationship between making progress towards verifying a linear-time 
property and finding a violation of its negation. First, we formalize that a search has not found a violation 
of a linear-time property. 

Definition 20 The search T of the PTS y has not found a violation of the linear-time property if there 
exists a PTS 5^' which extends T of y such that e\=yi <^ for all e G Execy. 

This definition is slightly stronger than the one given in [TT, Definition 7]. All results of fTP| remain 
valid for this stronger version. Next, we prove that if a search has made some progress towards verifying 
a linear-time property then that search has also found a violation of 0. 

Proposition 21 Let T be a search of the PTS y and let (j) be a linear-time property. Ifprogy{T, -i0 ) > 
then T has found a violation of<^. 

Proof By the definition of prog, jU.y/(^^t(r)) > for each PTS .9" which extends T of y. Hence, 
(r) / 0. Therefore, there exists eeT* such that B^, / and Me' G B'^, : e' \= y> Hence, e' ^ ^ 
and e' G Exec^'. Therefore, T has found a violation of ^. □ 

The reverse implication does not hold in general, as shown in the following example. 
Example 22 Consider the PTS 




2 



Assume that the state so satisfies the atomic proposition a and the state si does not. Consider the linear- 
time property Da and the search {too}. Note that ?oo® ^ -iQa and, hence, {?oo} has found a violation of 
-^^a. Also note that Tpmgy[{tQQ}, 'da) = 0. 

We conjecture that the reverse implication does hold for safety properties (see, for example, ||2l 
Definition 3.22] for a formal definition of safety property). However, so far we have only been able to 
prove it for invariants. 

Proposition 23 If the search T of the PTS y has found a violation of the invariant <\) then 
prog.y(r,^0)>O. 

Proof For every PTS y' that extends T , e Da for some e G Execy/. Hence, e = eftei> for some 
Cf G T* npref(Exec,y/) and f G T such that a label,^/(source,y'(f)). Therefore, for all e' G By,, we 
have that e' |= yi and B'^^, / 0. Hence, \i yiiffyl) > and, therefore, prog^'(r, -iDa) > 0. □ 

4 A Positive Fragment of LTL 

Next, we introduce a positive fragment of linear temporal logic (LTL). This fragment lacks negation. In 
Section |5] we will show how to compute the progress measure for this fragment. 

Definition 24 The logic LTL^ is defined by 

::= true | false | a | A | V | O0 \ <l>i'^ h \ h 

where a G AP. 
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The grammar defining LTL+ is tlie same as tlie grammar defining the logic PNF introduced in ||2j 
Definition 5.23], except that the grammar of LTL+ does not contain -la. For each LTL formula, there 
exists an equivalent PNF formula (see, for example, [2. Section 5.1.5]). Such a result, of course, does 
not hold for LTL+. 

A property of LTL+ that is key for our development is presented next. 

Proposition 25 For all LTL+ formulae and a e (2^^)*, a©*" |= (j) ijfMp G {2^^)'^ : op \= 0. 

Proof We prove two implications. Let be a LTL+ formula and let a G (2^^^)*. Assume that 
Vp G (2^^)® : ap 1= 0. Since 0" G (2^^)®, we can immediately conclude that a0® |= 0. 

The other implication is proved by structural induction on (p. Let a G (2^^^)*. We distinguish the follow- 
ing cases. 

• In case (p = true, clearly Vp G (2'^^)'" : ap \= <p and, hence, the property is satisfied. 

• In case (p = false, obviously a0® N ^ satisfied and, therefore, the property holds. 

• Let = a. If 00^" 1= (p, then |a| >0 and a G a[0] and, hence, Vp G (2-*^)® : ap |= 0. 

• Let = 01 A 02- Assume that a0® |= 0. Then a0'*' |= 0i and a0® |= 02- By induction, 
Vp G (2^^)® : ap 1= 01 and Vp G (2^^)® : ap |= 02- Hence, Vp G (2^^)" : ap |= 0. 

• The case = 01 V 02 is similar to the previous case. 

• For O*/* we distinguish the following two cases. Assume |a| = 0. Suppose a0® |= O0- Then 
0'"[1 . . .] = 0'*' 1= 0. By induction, Vp G (2'*^)'" : p ^ 0. Hence, Vp G (2'*^)'*' : p |= O0- 
Assume |a| > 1. Suppose a0'" |= 00- Then (a0'")[l...] = a[l...]0'" |= 0. By induction, 
Vp G (2'*^)'*' : a[l . . .]p 1= 0. Since a[l . . .]p = (ap)[l . . .], we have that Vp G (2^^)® : ap |= O0- 

• Next, let = 01 02. Assume that a0™ |= 0. Then there exists some j > such that 

(a) (a0'") [/ . . .] 1= 01 for all < / < j and 

(b) (a0'>')[j...] 1=02. 

We distinguish two cases. Suppose 7 < |a|. From (a) we can conclude that for all < / < j, 
(a0")[/...] = a[/...]0'" 1= 01. By induction, Vp G (2'^^)^ : a[i...]p \= 0i. Since a[i...]p = 
(ap)[/. . .], we have that Vp G (2^^)'*' : (ap)[/. . .] |= 0i. From(b) we can deduce that (a0®) [7 • • •] = 
a[j . . .]0'>' 1= 02. By induction, Vp G (2^^)'» : o[j . . .]p |= 02. Since o[j.. .]p = (ap)[j . . .], we have 
that Vp G (2^^)® : (ap) [j . . .] |= 02. Combining the above, we get Vp G (2^^)® : ap |= 0i ^ 02. 
Suppose j > |a|. For < / < |a|, the argument for (a) is the same as above. For |a| <i<j, 
(a) simply says that 0® |= 0i, which, by induction, implies that Vp G (2'*^)'*' : p ^ 0i. Hence, 
Vp G (2'*^)™ : (ap) [/ . . .] 1= 01 for all < / < j. In this case, (b) means 0® \= 02, which, by induction, 
impUes that Vp G (2^^)® : p |= 02. Hence, Vp G (2^^)® : (ap)[j. . .] ^ 02. Combining the above, 
we obtain that Vp G (2"^^)® : ap |= 0i 02. 

• Finally, we consider 0i ^ 02. According to El page 256], 0i ^ 02 = ~'(~'0i ~'02) and 
^(01 02) = (^02) ^ (^01 A ^02). According to fT. page 252], 0i 02 = (0i 02) V 001. 
Hence, we can derive that 0i ^ 02 = (02 (0i A 02)) V 002. Therefore, proving that the property 
is satisfied by 00, combined with the proofs for A, V and ^ above, suffices as proof for 0i M 02. 
Thus, we consider 00. Suppose that od)"' ^ □0. Then (a0®)[7. . .] |= for all j > 0. We dis- 
tinguish two cases. For all < 7 < |a|, we have that (a0™) [7 . . .] = a[7' . . .]0® |= 0. By induction, 
Vp G (2^^)® : a[7. . .]p 1= and, hence, Vp G (2^^)'" : (ap)[7. . .] |= 0. 

For all 7 > |a|, we have that (a0'")[7. . .] = 0® ^ 0. By induction, Vp G (2-*^)® : p |= and, 
therefore, Vp G (2'^^)™ : (ap) [7 . . .] |= 0. Combining the above, we get Vp G (2'*^)® : ap |= 00. 

□ 
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The above result does not hold for all LTL formulae, as shown in the following example. 

Example 26 Consider the LTL formula -^a. Note that this formula is not equivalent to any LTL^ for- 
mula. Let G = e. Obviously, Q'^ |= -la, but it is not the case that Vp G (2'*^)'" : p |= -i<3 (just take a 
pG (2^'')'^ with a e p[0\). 

5 An Algorithm to Compute Progress 

To obtain an algorithm to compute the progress for the positive fragment of LTL, we present an alternative 
characterization of the progress measure. This alternative characterization is cast in terms of a PTS built 
from the search as follows. We start from the transitions of the search and their source and target states. 
We add a sink state, which has a transition to itself with probability one and which does not satisfy 
any atomic proposition. For each state which has not been fully explored yet, that is, the sum of the 
probabilities of its outgoing transitions is less than one, we add a transition to the sink state with the 
remaining probability. This PTS can be viewed as the minimal extension of the search (we will formalize 
this in Proposition [34]). The PTS is defined as follows. 



Definition 27 Let T be a search of the PTS 5^. The set S'y is defined by 

Sy = { somcey{t) \t eT}U{ target y{t) \t eT}U {sq}. 

For each s £ S^y, 

outy{s) = ^{prob.y (f) I ? G r A source,y' (?) = s}. 

The PTS S^T is defined by 

• Ty^ = ru{f, I 5G Aouty(5)< 1}U{?_l}. 

{source,y'(f) ift^T 
s ift = ts 

si_ ift = tj_ 

targety(f) ift G T 
s± ift =t± ort = ts 

prob^(0 ift£T 
proby^ {t) = { I - out.y^ (s) ift = t, 
1 ift = ti_ 

ifs = s± 

labels (i') otherwise 

The above definition is very similar to ifTTl Definition 10]. The main difference is that we do not have 
final states. 



• target y^(f) 



• labelyj-(s) 



Proposition 28 Let T be a search of the PTS 5^. Then the PTS .5^j extends T . 

Proof Follows immediately from the definition of S^j. □ 

Next, we will show that the PTS S^j is the minimal extension of the search T of the PTS 5^ . More 
precisely, we will prove that for any other extension y of T we have that \Ly^ ) < M^' ) ■ To 
prove this result, we introduce two new notions and some of their properties. 
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Definition 29 Let T be a search of the PTS 5^ and let ^ he a linear-time property. The set Ey,{T) is 
defined by 

E%{T) = {eG r* npref(Exec | Ve' € B'y : e' \=.y (j) }. 
The set Ey^ (T) is minimal among the Ey,{T) where y extends T. 

Proposition 30 Let the PTS y" extend the search T of the PTS 5^ . For any LTL+ formula <p, 
E%{T)<ZE%,{T). 

Next, we restrict our attention to those elements of Ey{T) which are minimal with respect to the 
prefix order. 

Definition 31 Let T be a search of the PTS 5^ and let ^ be a linear-time property. The set MEy{T) is 
defined by 

ME%{T) = {e G £^(r) \\e\>Q^ 3e' G B'P'^^ : e' }. 
Note that e G MEy{T) if and only if it belongs to Ey{T) and none of its prefixes belong to Ey{T). 

Proposition 32 Let the PTSs 5^' and .5^" extend the search T of the PTS 5^ and let ^ be a linear-time 
property. Then 

U ^^5^' = U ^"y- 

Proof Since M£'^/„(r) C E^y„{T), we can conclude that the set on the left hand side is a subset of the 
set on the right hand side. Next, we prove the other inclusion. We show that for each e G Ey„{T) there 
exists e G ME'y„{T) such that By, C By, by induction on the length of e. In the base case, \e\ = 0, 
then e G Ey„{T) implies e G MEy„{T) and, hence, we take e to be e. Let \e\ >0. We distinguish two 
cases. If 3e' G B'P ^1 : e' (j) then we also take e to be e. Otherwise, e[\e\ — 1] G E'^y„{T). Obviously, 
By, C B'^f' and, by induction, there exists a e G ME^y„{T) such that Byif' C □ 

Proposition 33 Let the PTSs 5^' and S^" extend the search T of the PTS 5^ , and let (j) be a linear-time 
property. IfEy,{T) '^Ey„{T) then 

^y"'i[j{B'y»\eGME%iT)})= £ M^K^'y")- (D 

eeME^^,{T) 

Proof We have that 

MEt^,{T) C E%,{T) [by definition] 
C E'^y,„[T) [by assumption] 
C pref(Exec^//) [by definition] 

Hence, for all e G ME^yi{T), we have that B^y,, G £ yti. Since the set T is finite, the set T* is countable 
and, hence, the set MEy,{T) is countable as well. Since a a-algebra is closed under countable unions, 
\J{By„ I e G MEy,{T) } G Ty. Hence, the measure /x.yw is defined on this set. 

To conclude |ljl, it suffices to prove that for all ^1,^2^ ME'^y,(T) such that e\ / e2, ei is not a prefix 
of e2, since this implies that 6^, and B^y, are disjoint. Towards a contradiction, assume that e\ is a 
prefix of ^2- Since Ve'j G B^y, : e\ \=yi and ei is a prefix of e2 and ei 7^ ez, it cannot be the case that 
3^2 £ S^'J*^'' : e'2 y^yi <p. This contradicts the assumption that e2 G MEy,{T). □ 
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Now, we are ready to prove that the PTS J^t is the minimal extension of the search T of the PTS =5^. 
Proposition 34 Let the PTS y extend the search T of the PTS ^ and let (p be a LTL^ formula. Then 

^l.yA^%{T))<^ly,{m%{T)). 



Proof 



piy,{^%AT)) 

= ^iy,{[j{B':y.\eeE%{T)}) 

= I^^T (}J{By^ I e ME%^ (r) }) [Proposition|32 1 

= XI At^'r(^J^r) [Proposition [33j 

eeME%^{T) 

= IJ.yi{By,) [Proposition [TT) 

eeME%^(T) 

= l^y'(}J{By' I <^ 6 ME'^y^T) }) [Proposition |30] and [33| 
= Hy,(}J{B'y.\e£E'^^AT)}) [Proposition[32) 
< At y'([j{By' I e G £j„(r) }) [Proposition[30) 

= ^y>i^%{T)) 



□ 



The above proposition gives us an alternative characterization of the progress measure. 
Theorem 35 Let T be a search of the PTS and let ^ be a LTL^ formula. Then 

Wo%y[T.^)=\iyA^y,{^))- 
Proof This is a direct consequence of the definition of the progress measure and Proposition [34] □ 



Hence, in order to compute prog,5^(r, 0), it suffices to compute the measure of =^^,^(7). Next, we 
will show that the latter is equal to the measure of the set of execution paths of that satisfy 0. The 
proof consists of two parts. First, we prove the following inclusion. 

Proposition 36 Let T be a search of the PTS and let ^ be a linear-time property. Then 

e G Exec.yv I e |=yv }• 

Proof Let e G SB^^T). Then e G B''^^ for some e' G T* such that Ve" G B''^^ : e" \=y-,^. Hence, 
e \=yj (j). □ 

The opposite inclusion does not hold in general, as shown in the following example. 

Example 37 Consider the PTS y 

1 

2 _ 1 

so^'^Y^si 

2 
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Consider the search {foo}- Then the PTS S^j can he depicted by 



2 



Assume that the state sq satisfies the atomic proposition a. Hence, foo^ \= Da. By construction, the 
state Sj^ does not satisfy a. Therefore, t^o'^ 

However, we will show that the set { e G Exec^j. \ e \=y.j, (^}\ (T) has measure zero. In the 
proof, we will use the following proposition. 

Proposition 38 Let T be a search of the PTS 5^ and let ^ be a linear-time property. Assume that T has 
not found a violation of ^. Then for all e G PlExec^yj-, e \=yj ^. 

Proof Let e £T"'n Exec y^. Since T has not found a violation of 0, by definition there exists a PTS ^5^' 



that extends T of ^ such that e' \=yi for all e' G Exec y. Then e £ Execyi n by Proposition 10 ^b). 



because y and J>^t both extend T. Hence, e \=y' 0. Therefore, from Proposition 15 we can conclude 



that e \=yj . □ 

Proposition 39 Let T be a search of the PTS 5^ and let ^ be a LTL+ formula. If T has not found a 
violation of then 

lly, ({ e G Exec,^, | e ^y, ^}\^%^ (T)) = 0. 

Proof To avoid clutter, we denote the set G Execy-j. | e \=yj ^y^{T) by Z. 

First, we show that Z C r™. Assume that e G Z. Towards a contradiction, suppose that e T®. From 

the construction of we can deduce that e = e't^tx^ for some e' G T* . Let trace^^(e') = a. Then 



trace.^j,(e) = a0^. Since e \Ne, have that e \=y^ <p and, hence, a0® |= 0. By Proposition 25 



Vp G (2^^)"' : op 1= (j). Hence, Ve" G B'^^ : e" \=y^ (j). Since e G B"^^, we have that e G ^^^{T), which 
contradicts our assumption that e £ Z. 

Next, we show that each state in {targety^(e) | e G pref(Z)} is transient. Roughly speaking, a state s 
is transient if the probability of reaching s in one or more transitions when starting in s is strictly less 
than one (see, for example, [1 , Section 7.3] for a formal definition). It suffices to show that each state in 
{target yj^{e) \ e G pref(Z) } can reach the state sj_, since in that case the probability of reaching sj_ and, 
hence, not returning to the state itself, is greater than zero. 

Since T has not found a violation of 0, we can conclude from Proposition 38 that e \=yj ^ for all e G T'^. 



Hence, from the construction of S^j we can deduce that if e ^yj ^ then e and, hence, e reaches s^. 
Let e G pref(Z). Hence, there exists e' G such that e' y^y'j 0. Therefore, e' reaches s\^ and, hence, 
target^j, (e) can reach s^. 

Since Z C r™, the set { target | e G pref(Z) } is finite. According to ijU page 223], the probability 
of remaining in a finite set of transient states is zero. As a consequence, the probability of remaining in 
the set { target,5^/ (e)\e £ pref (Z) } is zero. Hence, we can conclude that \lyj (Z) = 0. □ 

From the above, we can derive the following result. 

Theorem 40 Let T be a search of the PTS and let ^ be a LTL+ formula. IfT has not found a violation 
ofcj) then 

^y,{^%^{T)) = ^ly,{{ee ExtCy^ \e^y,(P}). 



44 



Measuring Progress of Probabilistic LTL Model Checking 



Proof 

< pLy'j ({ e € Execy^ I e \=yj }) [Proposition 36 and iiy^ is monotone] 



lXy-^{^^yjA^)) + }Xyi^{{e € Execy^ \ e^y^ }\^yj.{T)) [Proposition [36] and pLy^ is additive] 
M yj {^yj {T)) [Proposition[39 1 

□ 



Combining Theorem 35 and 00) we obtain the following characterization of the progress measure. 



Corollary 41 Let T be a search of the PTS and let ^ he a LTL+ formula. If T has not found a 
violation of then 

pmgy{T, ) = }Xy^ ({ e G Exec.^^ | e |=.y^ }) . 
Proof Immediate consequence of Theorem [35] and [40] □ 



How to compute }jLy:j.{{e G Exec^^, | e \=yj 0}) can be found, for example, in lU Section 3.1]. 
Computing this measure is exponential in the size of and polynomial in the size of T . 

6 An Algorithm to Efficiently Compute a Lower Bound of Progress 

The algorithm developed in the previous section to compute prog,y(r, ^) is exponential in the size 
of 0. In this section, we trade precision for efficiency. We present an algorithm that does not com- 
pute prog,y'(r, 0), but only provides a lower bound in polynomial time. This lower bound is tight for 
invariants. However, we also show an example in which the lower bound does not provide us any infor- 
mation. 

Next, we show that subsets of Exec.y can be characterized as countable intersections of countable 
unions of basic cylinder sets. For A C Exec^ and « € N, we use A[n] to denote the set {^[w] | e G A}, 
where e[n] denotes the execution path e truncated at length n. We prove the characterization by showing 
two inclusions. The first inclusion holds for arbitrary subsets of Exec y. 

Proposition 42 For PTS J^, let A C Execy. Then 

AC n u 

neNeeA[n] 

Proof Let e' G A. It suffices to show that 

e'G [j B^y (2) 

eeA[n] 

for all « G N. Let « G N. To prove ([2]l, it suffices to show that e' G By for some e G A[?i]. Since e' G A, 

— '\ 1 

we have that e'[n] G A[?i]. Because e'[n] is a prefix of e' and e' G Exec y, we have that e' G By , which 
concludes our proof. □ 

The reverse inclusion does not hold in general. In some of the proofs below we use some metric 
topology. Those readers unfamiliar with metric topology are referred to, for example, JH. To prove the 
reverse inclusion, we use that the set is closed. 
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Proposition 43 For PTS ,y, let A C Exec.y . If A is closed then 

n U B'y^^- 

Proof Let e' G HnGN UeeA[«] ^j?"- Then e' G UeeA[n] ^"y for « G N. Hence, for each « G N there exists 
a e,j G A[n] such that e' G B^. Thus, for each « G N there exists a e', G A such that e' G S^'"' and, hence, 
^^[w] is a prefix of e' . 

We distinguish two cases. Assume that for some « G N, e'^[n\ = e',. Then e', is a prefix of e' . Since also 
e' , e', G Execy, we can conclude that / = e',. Since G A we have that e' G A. 

Otherwise, e'^[n] ^ e'^ for all n G N. Since also is a prefix of e', we can conclude that e'^[n] = e'[n\. 
Let the distance function d : (pref(Exec y) UExecy) x (pref(Exec.y ) UExecy ) — )• [0, 1] be defined by 
d{ei,e2) = inf{2^" | ei[n] = e2[n\}. Then, d{e'^,e') < 2^", that is, the sequence (e^)^ converges to e'. 
Because all the elements of the sequence (e^)„ are in A and A is closed, we can conclude that the limit e' 
is in A as well (see, for example, ||8] Proposition 3.7.15 and Lemma 7.2.2]). □ 

PTSs that extend a particular search assign the same measure to closed sets of execution paths con- 
sisting only of explored transitions. 

Proposition 44 Let the PTS .5^' extend the search T of the PTS ^ and let A C T'*' nExec^'. If A is 
closed then /x^ (A) = /i^/(A). 

Proof Obviously, for all e G T* and t € T, we have By, 5 By. As a consequence, [jeeA[n\^y' 5 
[jeeA[n+\]By for all n G N. Furthermore, ^iy{\JeeA[o]By) = ^iy{By) = 1 and, hence, lly{\JeeA[GiB'y) 
is finite. Since a measure is continuous (see, for example, fi. Theorem 2.1]), we can conclude from the 
above that 



A^^l n u =;iT^ u ^y 

,neNeeA[«] / \eeA\n\ 



(3) 



Therefore, 



jJiy{A) = I Pi [jSy^ [Proposition|42]and|43) 

,neNeeA[n] 



li^M^ U By 



neN 



lim ^ Hy^By) [a measure is countably additive] 



neN 



eeA\n\ 



t,...t„eA[n]l<i<n 

lim y TT proby/(f;) extends r of ^] 

ti...t„eA[n] l<i<n 

IJLyi(A) [by symmetric argument] . 



□ 
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Hence, the PTSs ^ and S^t assign the same measure to the closed set of those execution paths 
consisting only of explored transitions. 

Corollary 45 Let T be a search of the PTS y. Then iJ.y{T'" nExecy) = iJ.,y^{T"' nExec,y^). 

Proof Since the sets Exec y and are closed, their intersection is also closed (see, for example, lEl 
Proposition 3.7.5]) and, hence, the result follows immediately from Proposition |44] and [TOl^b). □ 

Now we can show that the measure of the set of execution paths consisting only of explored transi- 
tions is a lower bound for the progress measure. 

Theorem 46 Let T bea search of the PTS and let ^ be a LTL^ formula. IfT has not found a violation 
ofcj) then 

Ai^, (r"* n Exec^^ ) < prog^ (r, ) . 



Proof 



jXy'^{T'^r\Exec 



.9-7) 



< Hyj. {{e £ Exec.j/^ \ e \=y^ (f)}) [Proposition[38 1 
= prog^(r,0) [Corolla:7|4TJ 



□ 



From the construction of we can conclude that }JLy^{T"^ f\E\ecy^) is the same as 
IJ.y^{{e £ Exec I e does not reach s±}), which is the same as I — Hy^{{e £ Exec \ e reaches s± }). 
The latter can be computed in polynomial time using, for example, Gaussian elimination (see, for exam- 
ple, im Section 10.1.1]). This algorithm has been implemented and incorporated into an extension of the 
model checker JPF iflOl . While JPF is model checking sequential Java code which contains probabilistic 
choices, our extension also keeps track of the underlying PTS. The amount of memory needed to store 
this PTS is in general only a small fraction of the total amount of memory needed. Once our extension of 
JPF runs almost out of memory, it can usually free enough memory so that the progress can be computed 
from the stored PTS. 

As was shown in [1 1 , Theorem 4], the above bound is tight for invariants. 

Proposition 47 If the search T of the PTS 5^ has not found a violation of invariant then 

^yAT'"r\ Exec^v ) = prog^ (r, ^ ) . 

In the example below, we present a search of a PTS for a LTL+ formula of which the progress is one 
whereas the bound is zero. In this case, the bound does not provide us any information. 

Example 48 Consider the PTS 

So — 



Assume that the state si satisfies the atomic proposition a. Consider the linear-time property Qa and 
the search {toi 
M^„„,,(0) = O. 



the search {foi}. In this case, we have that progy ({foi}, O*^) = 1 ''^■^{'oi} ^^^''^^^ ^''''^'^'^{'oi} 
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7 Conclusion 

Our work is based on the paper by Zhang and Van Breugel lITTll . The work by Pavese, Braberman and 
Uchitel [6| is also related. They aim to measure the probability that a run of the system reaches a state 
that has not been visited by the model checker. Also the work by Delia Penna et al. fT| seems related. 
They show how, given a Markov chain and an integer /, the probability of reaching a particular state s 
within / transitions can be computed. 

As we have seen, there seems to be a trade off between efficiency and accuracy when it comes to 
computing progress. Our algorithm to compute progy(r, 0) is exponential in the size of the LTL+ 
formula <p and polynomial in the size of the search T. We even conjecture (and leave it to future work to 
prove) that the problem of computing progress is PSPACE-hard. However, in general the size of the LTL 
formula is small, whereas the size of the search is huge. Hence, we expect our algorithm to be useful. 

Providing a lower bound for the progress measure can be done in polynomial time. As we have 
shown, this bound is tight for invariants. Invariants form an important class of properties. Determining 
the class of LTL+ formulae for which the bound is tight is another topic for further research. 

The approach to handle the positive fragment of LTL seems not applicable to all of LTL. We believe 
that a different approach is needed and leave this for future research. 
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